3-d model providing device

ABSTRACT

A three-dimensional model providing apparatus in which a customer can easily modify the input three-dimensional data. The apparatus comprises an input section ( 10 ) for inputting three-dimensional data, a server section ( 12 ) for managing the processes on the three-dimensional data from input to output, a providing section ( 14 ) for providing three-dimensional data, a communicating section for communicating the three-dimensional data between the input section, server section, and providing section, a receiving section ( 11 ) for receiving instruction contents from the customer, and a converting section ( 13 ) for modifying, processing, or converting the three-dimensional data according to the instruction contents.

This application is the national phase of international applicationPCT/JP00/00930 filed Feb. 18, 2000 which was not published under PCTArticle 21 (2) in English.

TECHNICAL FIELD

The present invention relates to a three-dimensional model providingapparatus for capturing an image of a target, generating athree-dimensional model of the target, and providing the data to acustomer in a desired content.

BACKGROUND ART

For some years, three-dimensional scanners which can be used to obtainthree-dimensional shape data and color information of various types ofobjects have been known. Various software for capturing thethee-dimensional data into a computer and converting it to a desireddata format are also available. It is also known to output onto adesired recording medium and communicate via a network using peripheraldevices of a computer.

However, because many of the three-dimensional scanners are large andexpensive, they represent a major investment to purchasers, who generalpurchase a three-dimensional scanner for a special purpose. There has,therefore, been no means for ordinary users to scan or obtainthree-dimensional data. Moreover, among the three-dimensional scanners,a method for obtaining three-dimensional shape data by scanning laserlight onto a target in particular requires a long period of time forscanning and requires the object to be still for the entire duration ofthat scanning period. Because of this, it was difficult to obtainthree-dimensional data of, for example, a living human being or otherliving or animated target.

Moreover, it is not always safe to irradiate laser light onto a person.

Furthermore, when a customer wishes to modify the inputthree-dimensional data, there has heretofore existed no system for thecustomer to easily transmit the three-dimensional data along with amodification request (desire) of the customer to a specializedmodification processor.

DISCLOSURE OF THE INVENTION

The present invention was conceived to solve the above described problemand one object of the present invention is to provide athree-dimensional model providing apparatus which enables a generalcustomer to freely create and use a three-dimensional model.

In order to achieve the object above, according to one aspect of thepresent invention, there is provided a three-dimensional model providingapparatus for supplying three-dimensional data of a target to acustomer, the apparatus comprising an input section for inputtingthree-dimensional data; a server section for managing thethree-dimensional data; a providing section for providing thethree-dimensional data; and a communicating section for transceiving thethree-dimensional data between the input section, server section, andproviding section.

According to another aspect of the present invention, it is preferablethat the three-dimensional model providing apparatus further comprises areceiving section for receiving instruction contents from the customer;and a converting section for modifying, processing, or converting thethree-dimensional data based on the instruction contents.

According to another aspect of the present invention, it is preferablethat the three-dimensional model providing apparatus further comprisesan output section for confirming, visually or by audio, the inputthree-dimensional data or modified or processed three-dimensional data.

According to another aspect of the present invention, it is preferablethat the output section is capable of three-dimensional display.

According to another aspect of the present invention, it is preferablethat the input section produces three-dimensional shape data based onimage data of the target captured from a plurality of directions.

According to another aspect of the present invention, it is preferablethat the input section produces color data associated with thethree-dimensional shape data based on image data of the target capturedfrom a plurality of directions.

According to another aspect of the present invention, it is preferablethat the input section produces three-dimensional shape data based onimage data of the target captured from a plurality of directions by aplurality of cameras.

According to another aspect of the present invention, it is preferablethat the input section produces color data associated with thethree-dimensional shape data based on image data of the target capturedfrom a plurality of directions by a plurality of cameras.

According to another aspect of the present invention, it is preferablethat the input section includes a light irradiating section forirradiating light onto the target.

According to another aspect of the present invention, it is preferablethat the input section comprises a light projection section forprojecting light; a light reflecting section for reflecting lightincident from the light projection section and projecting onto thetarget; a light receiving section for receiving light projected onto thetarget and reflected therefrom; and a modeling section for producingthree-dimensional data of the target based on the received light data.

According to another aspect of the present invention, it is preferablethat the light projection section is a surface light source forprojecting a pattern light.

According to another aspect of the present invention, it is preferablethat the input section comprises a light projection section forprojecting light onto the target; a target image reflecting section forfurther reflecting light projected onto the target and reflectedtherefrom; a light receiving section for receiving the target imagereflected at the target image reflecting section; and a modeling sectionfor producing three-dimensional data of the target based on the receivedlight data.

According to another aspect of the present invention, it is preferablethat the instruction content from the customer received at the receivingsection is a data format used when the three-dimensional data isprovided by the three-dimensional model providing apparatus.

According to another aspect of the present invention, it s preferablethat the instruction content from the customer received at the receivingsection is modification and processing content to be applied to thethree-dimensional data by the three-dimensional model providingapparatus.

According to another aspect of the present invention, it is preferablethat the instruction content from the customer received at the receivingsection is a data providing method designating the providing method ofthe three-dimensional data by the three-dimensional model providingapparatus.

According to another aspect of the present invention, it is preferablethat the receiving section comprises a received content output sectionfor outputting the instruction content from the customer and theidentification information for identifying the customer.

According to another aspect of the present invention, it is preferablethat the received content output section comprises one of adisplay-at-display section, an output-by-printing section, and anoutput-to-recording-medium section.

According to another aspect of the present invention, it is preferablethat the converting section converts the three-dimensional dataaccording to the data format.

According to another aspect of the present invention, it is preferablethat the converting section modifies, processes, or converts thethree-dimensional data according to the modification and processingcontents.

According to another aspect of the present invention, it is preferableto manage operation progress condition of the providing section and theconverting section, in addition to managing the three-dimensional data.

According to another aspect of the present invention, it is preferablethat the three-dimensional model providing apparatus further comprises acommunicating section, and the apparatus communicates, via thecommunicating section, the three-dimensional data, the instructioncontent from the customer, and identification information foridentifying the customer to/from another three-dimensional modelproviding apparatus having a function equivalent to the convertingsection.

According to another aspect of the present invention, it is preferablethat the three-dimensional model providing apparatus further comprises acommunicating section, and the apparatus communicates, via thecommunicating section, three-dimensional data, instruction contents fromthe customer, and/or identification information for identifying thecustomer to/from another three-dimensional model providing apparatushaving a function equivalent to the providing section.

According to another aspect of the present invention, it is preferablethat the providing section comprises a recording medium selectingsection capable of selecting an arbitrary recording medium from among atleast one type of recording medium which are provided to thethree-dimensional model providing apparatus in advance, and a recordingmedium writing section for writing three-dimensional data onto therecording medium.

According to another aspect of the present invention, it is preferablethat the providing section comprises a recording medium writing sectionfor writing the three-dimensional data onto a recording medium preparedby the customer.

According to another aspect of the present invention, it is preferablethat the providing section comprises recording medium mailing means formailing the recording medium.

According to another aspect of the present invention, it is preferablethat the providing section sends the three-dimensional data to aproviding section of another three-dimensional model providing apparatusvia the communicating section.

According to another aspect of the present invention, it is preferablethat the other three-dimensional model providing apparatus is anapparatus designated by the customer (referred to as a “designatedapparatus” hereinafter).

According to another aspect of the present invention, it is preferablethat the other three-dimensional model providing apparatus is a databaseserver for managing access by the customer, in addition to accumulatingand managing the three-dimensional data.

According to another aspect of the present invention, it is preferablethat the three-dimensional data and unique information of the customerare communicated to/from the designated apparatus based on the dataproviding method.

According to another aspect of the present invention, it is preferablethat the designated apparatus is capable of adding new information tothe unique information.

According to another aspect of the present invention, it is preferablethat the designated device is an entertainment device.

According to another aspect of the present invention, it is preferablethat the designated apparatus is an entertainment device and the newinformation is the score evaluated by the entertainment device as aresult of the entertainment.

According to another aspect of the present invention, it is preferablethat the designated apparatus is an entertainment device and the newinformation is information for updating the three-dimensional dataitself.

According to another aspect of the present invention, it is preferablethat the three-dimensional model providing apparatus further comprisesan information re-inputting section for re-reading the information withthe new information added at the designated apparatus.

According to another aspect of the present invention, it is preferablethat the designated apparatus is an entertainment device and thethree-dimensional data recorded in the recording medium is used by anentertainment program of the entertainment device.

According to another aspect of the present invention, it is preferablethat the designated apparatus is a database section and manages theinput three-dimensional model and entertainment program.

According to another aspect of the present invention, it is preferablethat the server section reads a model template of entertainmentcharacters from the database section, and the input three-dimensionalmodel is changed at the converting section based on the character modeltemplate.

According to another aspect of the present invention, it is preferablethat the server section reads an entertainment program from the databasesection, and the input three-dimensional model is incorporated into theentertainment program at the converting section.

According to another aspect of the present invention, it is preferablethat there exist a plurality of entertainment devices.

According to another aspect of the present invention, it is preferablethat the plurality of entertainment devices are capable of communicatingwith each other.

According to another aspect of the present invention, it is preferablethat the designated apparatus is a device connected to a networkincluding a computer.

According to another aspect of the present invention, there is provideda three-dimensional model providing apparatus for supplyingthree-dimensional data of a target to a customer, the apparatuscomprising an input section for inputting two-dimensional image data; aproducing section for producing three-dimensional data from thetwo-dimensional image data; a server section for managing thethree-dimensional data from input to output; a providing section forproviding three-dimensional data; and a communicating section forcommunicating the three-dimensional data between the input section,producing section, server section, and providing section.

According to another aspect of the present invention, it is preferablethat the two-dimensional image data is at least one portion of thetarget image data.

According to another aspect of the present invention, it is preferablethat the two-dimensional image data is image data captured in accordancewith a method for automatically producing three-dimensional data.

According to another aspect of the present invention, it is preferablethat the communicating section is capable of communicating with anexternal network.

According to another aspect of the present invention, there is provideda three-dimensional model providing apparatus for supplying athree-dimensional model to a customer, the three-dimensional modelobtained by combining at least two three-dimensional models, theapparatus comprising a database section capable of recording at leastone three-dimensional model data; a controlling section for controllingthe selection of three-dimensional model data recorded at the databasesection and the interface with a user; a data integrating section forintegrating the selected three-dimensional model data; and a displaysection for displaying the three-dimensional model data.

According to another aspect of the present invention, it is preferablethat the data integrating section changes the shapes of at least twothree-dimensional model data to be integrated and then integrates thedata.

According to another aspect of the present invention, it is preferablethat the three-dimensional model providing apparatus further comprises acolor changing section for changing color of the selectedthree-dimensional model data.

According to another aspect of the present invention, it is preferablethat the display section is a display apparatus capable ofthree-dimensional display.

According to another aspect of the present invention, it is preferablethat the three-dimensional model providing apparatus further comprises acommunicating section capable of communicating with a database sectionvia a network, and the three-dimensional model data is recorded in thedatabase section.

According to another aspect of the present invention, it is preferablethat a plurality of three-dimensional model data recorded at thedatabase section are read and the read three-dimensional model data areintegrated to produce three-dimensional data.

According to another aspect of the present invention, it is preferablethat the three-dimensional model providing apparatus further comprisesan optional function display section for allowing a customer to selectan optional function.

According to another aspect of the present invention, it is preferablethat the three-dimensional model providing apparatus further comprisinga three-dimensional data modifying section for modifying the inputthree-dimensional data based on the selected optional function.

According to another aspect of the present invention, it is preferablethat the optional function adds a pair of glasses to the inputthree-dimensional data.

According to another aspect of the present invention, it is preferablethat the optional function changes the hair style of or adds hair to theinput three-dimensional data.

According to another aspect of the present invention, it is preferablethat the optional function assigns a color to each section.

According to another aspect of the present invention, it is preferablethat the optional function adds an ornament to the inputthree-dimensional data.

According to another aspect of the present invention, it is preferablethat the optional function changes the shape proportion of thethree-dimensional data.

In order to achieve the object described above, according to anotheraspect of the present invention, there is provided a three-dimensionalmodel providing method for supplying three-dimensional data of a targetto a customer, the method comprising the steps of inputtingthree-dimensional data; managing the three-dimensional data at a serversection; providing the three-dimensional data at a providing section;and transmitting/receiving the three-dimensional data between the inputsection, server section, and providing section, via a communicatingsection.

According to another aspect of the present invention, it is preferablethat the three-dimensional model providing method further comprises astep of receiving instruction contents from the customer at a receivingsection, and a step for modifying, processing, or converting thethree-dimensional data based on the instruction contents at a convertingsection.

According to another aspect of the present invention, it is preferablethat the three-dimensional model providing method further comprises thestep of confirming, visually or by audio, the input three-dimensionaldata or modified or processed three-dimensional data at an outputsection.

According to another aspect of the present invention, it is preferablethat the output section performs three-dimensional display.

According to another aspect of the present invention, there is provideda three-dimensional model providing method for supplyingthree-dimensional data of a target to a customer, the method comprisingthe steps of inputting two-dimensional image data; producingthree-dimensional data from the two-dimensional image data; managing thethree-dimensional data from input to output at a server section;providing three-dimensional data at a providing section; andtransmitting/receiving the three-dimensional data between the inputsection, producing section, server section, and providing section, via acommunicating section.

According to another aspect of the present invention, it is preferablethat the two-dimensional image data is at least one image data of thetarget.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram showing a structure of one embodiment of thepresent invention.

FIG. 2 is a block diagram showing a structure of the input section shownin FIG. 1.

FIG. 3 is a conceptual diagram showing a structure of the input section.

FIG. 4 is a block diagram showing a structure of the receiving sectionshown in FIG. 1.

FIG. 5 is a block diagram of a flow of a recording medium at theproviding section.

FIG. 6 is a block diagram showing a flow of three-dimensional data atthe providing section.

FIG. 7 is a block diagram showing a structure when three-dimensionaldata is used with an entertainment device or the like.

FIG. 8 is a block diagram showing a structure when three-dimensionaldata is produced from two-dimensional image data.

FIG. 9 is a block diagram showing a structure for realizing anintegration function of three-dimensional data.

FIG. 10 is a diagram showing calibration.

FIG. 11 is a diagram showing a structure for projecting a randompattern.

FIG. 12 is a diagram showing an example of a random pattern.

FIG. 13 is a diagram showing a structure of a three-dimensional dataproducing apparatus according to another embodiment of the presentinvention.

FIGS. 14( a) and (b) are diagrams explaining positions of anillumination light source.

FIGS. 15( a) and (b) are diagrams explaining a laser irradiating device.

FIGS. 16( a) and (b) are diagrams explaining camera positions foravoiding blind spots.

FIG. 17 is a diagram showing a structure of a three-dimensional dataproducing apparatus according to another embodiment of the presentinvention.

FIG. 18 is a diagram showing a flow of data input.

BEST MODE FOR CARRYING OUT THE INVENTION

Embodiments of the present invention will now be described referring tothe drawings. FIG. 1 shows a structure of a three-dimensional modelproviding apparatus according to one embodiment of the presentinvention.

A three-dimensional data input section 10 captures an image of a targetsuch as a person or the face of a person, and produces three-dimensionaldata. The three-dimensional data input section 10 may be, for example, athree-dimensional data inputting apparatus as shown in FIGS. 2 and 3. Animage data input section 20 captures an image of the target and producesimage data. The image data input section 20 includes a plurality offixed color CCD cameras, as shown in FIG. 3, and obtains color imagedata of the target at a plurality of different positions.

In the following descriptions, the term “three-dimensional shape data”refers to data of the three-dimensional shape information of the target,and may, in some cases, include the color information. Thethree-dimensional shape data is obtained by inputting an image at thedata input section and performing a modeling process at a modelingsection. When the data input section is a three-dimensional shapemeasurement device, the three-dimensional shape data may be obtained atthe data input section. Moreover, data which is obtained by changing,modifying, integrating, and/or processing the obtained three-dimensionalshape data is also referred to as three-dimensional shape data.

The color image data, including the color information, obtained by theimage data input section 20 is input to a modeling section 21. Themodeling section 21 includes a computer or the like, and performsthree-dimensional modeling based on the plurality of image data of thetarget captured at the plurality of different positions, to obtainthree-dimensional data of the target. The three-dimensional dataincludes the color data of the target. For the three-dimensionalmodeling, any method can be used such as, for example, a methoddisclosed in Japanese Patent Laid-Open Publication No. Hei 10-124704.

A display/output section 15 displays/outputs, visually or by audio,input three-dimensional data, confirmation of the reception process andreception content at a receiving section 11, confirmation of thethree-dimensional data after processing at a converting section 13, andinformation at a providing section 14 (to be described later). Thedisplay of the three-dimensional data is possible using a display whichis capable of three-dimensional display.

As shown in FIG. 4, the receiving section 11 includes one of a dataformat receiving section 40 for receiving a data format to be used whenthree-dimensional data is provided to the customer from thethree-dimensional model providing apparatus, a modification processcontent receiving section 41 for receiving the content of themodification processing of the data when providing the three-dimensionaldata, and a providing method receiving section 42 for receiving themethod for providing the three-dimensional data, and a received contentoutput section 43 for outputting the received contents and customeridentification information required when providing the data. Thereceived content output section 43 includes a display-at-displaysection, an output-by-printing section, and anoutput-to-recording-medium section. The display-at-display section canalso be a display section 15.

The data format at the data format receiving section 40 can be, forexample, general three-dimensional model description format, a gamecompatible format which can be input into a game program, a networkcompatible format where data amount is compressed for networkcommunication, or a security compatible format used for personalauthentication. When a security compatible format is selected, personalinformation of the customer is received and the three-dimensional dataincluding the personal information is provided. In such a case, it ispreferable to not only encrypt the three-dimensional data of the target,but also to include the personal information for encryption.

The modification process content receiving section 41 designates thecontent of the modification processing desired by the customer such as,for example, modification of the input three-dimensional data for betterlooks, deformation of the input data so that the characteristics areemphasized, or modification of the input data so that it is moreanimation-like.

At the providing method receiving section 42, the customer designatesthe method for providing the input three-dimensional data to thecustomer. Examples of such methods include a recording medium preparedat the three-dimensional model providing apparatus in advance and arecording medium prepared by the customer. In this case, because the agreat variety of recording mediums are currently in use, it ispreferable that the type of the input recording medium be detectedautomatically. It is also possible to provide the recording medium tothe customer by mail. In this case, the destination information such asthe address to which the medium is to be mailed is also received.

In addition, there also exists a method for directly transferring to thecomputer or the mobile terminal (for example, PDA, PHS, or portablephone) that the customer possesses, using communication means. In thiscase, the type of communication (protocol) and information of thecommunication destination (address) are received.

Moreover, there also exists a method for providing the three-dimensionaldata to the customer by transferring the data to a server on a networkand allowing the customer to access the server. In this case,identification information such as a number or a password foridentifying the customer when accessing can be received or issued at aproviding section 14 of the three-dimensional model providing apparatus.

A server section 12 manages the data from input to output of thethree-dimensional data, schedules the processes, and/or prepares datasuited for the method for providing the thee-dimensional data to thecustomer.

The converting section 13 includes at least one of a modificationprocessing section for modify processing according to the contentdesignated by the customer at the modification process content receivingsection 41 or a data converting section for converting thethree-dimensional data to be provided into a data format received fromthe customer at the data format receiving section 40. The convertingsection 13 can be placed within the three-dimensional model providingapparatus or within a three-dimensional model providing apparatuslocated at a different location by communication means such as anetwork, in which case the three-dimensional data is transmitted viacommunication. The apparatus at different location in this case can be aserver, in which case, a particular processing operator accesses theserver, transmits and receives data, and processes the data. Moreover,the modification processing section and the data converting section canalso be independently provided. In such a case, it is preferable toallow the data to be interchanged by a network or a recording medium.

FIG. 5 shows a sequence of processes at the providing section 14 such aswriting the three-dimensional data onto the recording medium.

A desired recording medium is selected at a recording medium selectingsection 51 from among recording media pre-provided at a recording mediumstoring section 50. The final data is written onto the selectedrecording medium at a three-dimensional data writing section 52. Byoutputting the recording medium at a recording medium output section 54,the recording medium can be provided to the customer. Alternatively, arecording medium prepared by the customer can be input to a recordingmedium input section 53, and, by writing the three-dimensional data, therecording medium can be provided to the customer. During this process,it is preferable to automatically identify the type of the inputrecording medium at the recording medium input section 53.

Examples of the recording medium include a magnetic card, a card typememory, a floppy disk, a magnetic disk, CD-R/RW, and a security card forauthentication. It is also possible to provide these recording media bymail or by a door-to-door delivery service. In such a case, it ispreferable to send the data to a providing section near the maildestination via a network, output the recording medium there, and mailor allow the customer to return and collect the recording medium at thatlocation, because in any of these ways delivery costs can be reduced.

FIG. 6 shows a flow of the recording medium or the three-dimensionaldata.

A communicating section 61 of the three-dimensional model providingapparatus includes a communication function section 62 of infrared andRS-232C, and transmits the three-dimensional data to the mobile terminalof the customer. Also, the communicating section 61 connects to anetwork via, for example, an Ethernet, a phone line, ISDN, satellitecommunication, or digital broadcast, and transmits the three-dimensionaldata to the destination designated by the customer. The destination thatcan be designated by the customer includes a computer at home and aserver on a network. The transmission method includes electronic mailand file transfer. When the data is to be accumulated at a server on anetwork by transferring an electric mail or file, the customer canreceive, at the receiving section 11, identification information foridentifying the customer when the customer accesses the server from hiscomputer or mobile terminal 63, or such identification information canbe issued at the providing section 14 and provided to the customer.

FIG. 7 shows a flow of data such as the three-dimensional data andentertainment program when the provided three-dimensional data is usedat various entertainment devices in a case where a game compatibleformat is designated at the data format receiving section 40.

Here, a target input by a customer, for example, a face or a body of thecustomer, or a character created by the customer can be input to athree-dimensional data input compatible entertainment device as acharacter for the game and play the game. The three-dimensional data isoutput by a method designated by the customer at the providing methodreceiving section 42 or output by a providing method suited forconnection configuration of the entertainment device. This configurationalso enables fitting of the input three-dimensional data into the uniqueinput format of the entertainment device as a character bytransmitting/receiving with a database server for managing theentertainment program or character data, and entertainment usingthree-dimensional data of the target input by an entertainment deviceother than the three-dimensional data input compatible entertainmentdevice by incorporating into the program itself.

As a result of the entertainment at the entertainment device, theobtained score or the results and score can be transmitted/received witha three-dimensional model providing apparatus via a recording medium ora communication medium. By reflecting the obtained score or the likewhen the three-dimensional data is modified or changed, it is possibleto improve the fun of the entertainment. For example, the character'sface can be beautified or the muscles can be strengthened when a highscore or good results are achieved.

In addition, by providing in advance materials in the apparatus that canbe processed and that are needed for creating a character, it ispossible to provide data satisfying the customer's specific demands.

FIG. 8 is a diagram showing a case where three-dimensional data isproduced from two-dimensional image data of the target and then providedto the customer. The customer sends the two-dimensional image dataobtained by capturing the image of the target for whichthree-dimensional data is desired from a two-dimensional image datainput section 81 to a server section 82 via a recording medium or acommunication medium such as a network, and inputs into thethree-dimensional model providing apparatus. The input two-dimensionalimage data is sent to a producing section 83 where the three-dimensionaldata is produced. The three-dimensional data produced at the producingsection 83 is provided to the customer at a providing section 84. Theproducing section 83 can be connected to the server section 82 via, forexample, a network where the transmitted two-dimensional data isdisplayed and a specialized operator can manually create thethree-dimensional data based on the two-dimensional image data. It ispossible to realize an effective three-dimensional model providingapparatus by managing the transmission/reception of the data and theprogress conditions for the operations of the producing section 83 atthe server section 82.

Efficiency can be further improved when the input two-dimensional imagedata is an image captured in a condition where the data can beautomatically converted into three-dimension, more specifically, a casewhere there are a plurality of images and the positional relationshipbetween the cameras which captured the images are known, due to thepossibility of automatic three-dimensional data production.

FIG. 9 shows a configuration for realizing an integrating function ofthe three-dimensional data according to another embodiment of thepresent invention. By selecting some of the three-dimensional data inputat the input section (including the three-dimensional data provided bythe customer) and from three-dimensional data registered in a databasesection, a plurality of three-dimensional data can be integrated whennecessary and the integrated result can be confirmed at a displaysection. For example, input three-dimensional data of a human headsection provided by the customer can be integrated withthree-dimensional data of a body wearing a piece of clothingpre-registered in the database section. During this process, by changingsome of the used three-dimensional data with a reference at one of thedata size, integration with overall good balance can be achieved.

As another example, input three-dimensional data of a human body with nocloth can be integrated with the three-dimensional data of a piece ofclothing in order to simulate dressing. During this process, by alteringthe cloth data to fit the human body, or by altering the human bodyaccording to the characteristic of the cloth, it is possible to simulatetrying-on of clothing. The three-dimensional data registered in thedatabase not only includes clothes, but also ornaments such as glassesand accessories, and styling elements such as hair style, makeup, andproportion. By selecting among these elements, the three-dimensionaldata can be arbitrarily modified.

By providing the database section on a network so that it can becommunicated with the other three-dimensional model providingapparatuses, the three-dimensional data can be managed at once,resulting in improvements in efficiency, and thus, increase in thenumber of three-dimensional data that are registered. By enablingregistration and management of the input three-dimensional data at thedatabase, the number of types can be further increased, resulting inmore convenient configuration.

“Structure of the three-dimensional data input section (image data inputsection and modeling section (refer to FIG. 2))”

The data input section 20 captures an image of a target using a camera.In order to obtain three-dimensional shape data, it is generallyrequired to obtain a plurality of image data from a plurality ofdirections for one immovable target. It is therefore preferable to affixa plurality of cameras at predetermined positions and simultaneouslycapture images of the target. It is however also possible to moveablymount a camera on a rail and obtain a plurality of image data by movingthe camera.

Reliable detection of the absolute positions of the plurality of fixedcameras are required, but in general this may be difficult with themechanical means. To this end, as shown in FIG. 10, it is preferable toplace a reference object with a predetermined pattern drawn in thecamera view and to perform calibration of camera position based on theimage data of the reference object. In this manner, the position of thecamera can be accurately detected with a simple mounting of camera. Itis preferable to perform the calibration periodically, e.g. on dailybasis or on weekly basis.

In order to accurately obtain the three-dimensional shape data of thetarget, each of the sections of the target must be seen differently froma plurality of directions. As shown in FIG. 11, it is preferable toproject a predetermined pattern onto the target using a projector. Inthis manner, an accurate three-dimensional shape data can be obtainedeven for sections that have uniform colors or where color differencesare small. The pattern can be removed from the color data by dataprocess or, alternatively, the color data can be obtained from thecaptured data in a case where the pattern is not projected. As theprojection pattern, a random pattern as shown in FIG. 12 can bepreferably used.

When three-dimensional data is obtained using such a random pattern, ahigh-precision modeling can be performed even for the convex sections byobtaining depth data using a stereo method with a plurality of cameras.

As the camera, it is preferable to use a camera with a wide-angle lens.This enables provision of the camera in a relatively small space.

The data input section 20 captures an image of a target by a camera andextracts the target area in the image. It is therefore preferable to usea constant background color to facilitate the clipping of the targetarea.

For example, it is preferable to provide a room with all four wallshaving a uniform color and place the target at the center section of theroom. In particular, when the target is a person, it is preferable toprovide a chair at the center section and let the person sit on thechair. Also, in a case of a person, the image from the back side isrelatively unimportant.

Thus, it is preferable to provide an entrance at one wall of the roomand let the person sit facing this entrance, while unifying the colorsof the walls except for the entrance side. For a person, it is alsopreferable to omit the image data from the back side.

When the target and the background have the same color, the clippingbecomes difficult. It is therefore preferable to allow a change in thebackground color (the color of the room walls). For example, byprojecting a light ray of predetermined color from outside using aprojector, the wall color can be changed.

Moreover, it is also preferable to change at least two colors in thebackground color, obtain image data at the two types of backgroundcolors, and clip the target area in the image by the sum of theextraction results at both background colors. In this manner, the targetarea can be reliably clipped for any target color.

Furthermore, it is preferable to provide illumination devices at thefour corners of the room and light up the room so that there will be noshadow on the target, in order to unify the illumination for the target.

“Others”

When a pet such as a dog is the target, it is necessary to restrain thepet. It is preferable to provide a chain at the image capturing station.In this case, by unifying the color of the chain and the background, itis possible to facilitate extraction of the target in the captured imagedata.

FIG. 13 shows a structure of a three-dimensional data producingapparatus at an input section 10 according to another embodiment of thepresent invention.

A light projection section 200 projects a random pattern, slit pattern,or coded pattern assigned on a panel 201 onto a target such as, forexample, a person or a person's face. The projected pattern light isprojected onto the target 203 after reflecting at a reflecting section202.

The target image with the pattern light projected is captured at animage input section 204 such as a CCD camera and the three-dimensionaldata of the target is produced at a modeling section 205 from aplurality of obtained images by the method already described.

When the distance from the light source of the light projection section200 to the target is small and the pattern is directly projected ontothe target, there will be sections where the pattern is not projecteddue to the shadow of the target itself. In other words, there will beportions which cannot be seen from the light source. In the case shownin FIG. 14( a), the pattern is not projected onto a part of the neck dueto the shadow of the chin.

Thus, the shape measurement of such part will either be very difficultor inaccurate.

In order to minimize such possibility of a blind spot, the distancebetween the light source and the target can be set at a sufficientlylarge distance, as shown in FIG. 14( b). However, this configurationwill result in a very large overall size of the apparatus.

Therefore, the projection pattern is projected onto the target afterbeing reflected at the reflecting section 202.

In this manner, it is possible to increase the distance of the lightpath from the light projection section 200 to the target whilepreventing the overall size of the apparatus from becoming large.

It is also possible to irradiate a laser ray in place of the patternlight from the light projection section 200 as described above. A laserray irradiating device has a basic structure as shown in FIG. 15.

Specifically, a two-dimensional scan is enabled in the configurationshown in FIG. 15( a) by changing the progress direction of the beam-typelaser ray using a galvano mirror or a polygon mirror.

In the configuration shown in FIG. 15( b), a two-dimensional scan isenabled by changing the beam-type laser ray into a slit light by acylindrical lens and changing the progress direction of the slit lightusing a galvano mirror or a polygon mirror.

Even in the above-described cases, if the distance to the target isinsufficient, the problem of the blind spot will be generated, as alsodescribed above. Therefore, a method for increasing the light path usinga reflecting section is useful from the view of reducing the blindspots.

Similarly, problematic blind spots are also generated during the inputof an object with the pattern projected. For example, as shown in FIG.16( a), when capturing an image of a pattern-projected object using aCCD camera, it is not possible to input the portion of the neck that isblocked by the chin for the same reason as described above. As shown inFIG. 16( b), the distance from the camera to the target can be set at asufficiently large distance using a lens with a longer focal length.However, this configuration will again result in a very large overallsize of the apparatus.

Thus, the image of the pattern-projected target reflected at thereflecting section is captured by the camera. In this manner, it ispossible to elongate the distance from the target to the camera whilepreventing the overall size from becoming too large. This configurationcan minimize blind spots.

FIG. 17 shows a structure of a three-dimensional data producingapparatus according to another embodiment of the present invention.

A plurality of data input sections 240 are present for capturing animage of a target from a plurality of directions and inputting imagedata. A data input section 240 includes an image input section 241. Asthe image input section 241, an analog or digital color CCD camera orthe like may be employed.

A portion of the data input sections includes a data input referenceparameter determining section 241 for suitably determining theparameters during the data input, such as, for example, white, balance,exposure, etc.

Each of the other data input sections, on the other hand, includes aparameter setting section 243 for setting the parameters to theparameters determined at the data input reference parameter determiningsection. At these data input sections, images are input based on the setparameters.

At a modeling section 244, three-dimensional data of the target iscalculated based on a plurality of images obtained at the data inputsections. The modeling process can be performed by, for example, themethod described in Japanese Patent Laid-Open Publication No. Hei10-124704.

The flow of the data input will now be described referring to FIG. 18.

The parameters such as the white balance and exposure are automaticallydetermined at one data input section (step S10). This can be easilyaccomplished using functions such as auto-white balance andauto-exposure which are commonly provided in electronic cameras.

The determined parameters are then notified to the other data inputsections (step S11).

At each data input section, the notified parameters are set at theinternal parameter setting section of the data input section (step S12).In other words, at these data input sections, the parameters are notautomatically set, but are set by the parameters notified from outside.

The images are input at all data input sections based on the setparameters (step S13).

In this manner, by setting the parameters determined at one data inputsection as common parameters and by inputting images at all data inputsections based on the common parameters, it is possible to input imageswith constant quality.

In contrast, if each data input section separately sets suitableparameters, different values will be set as the suitable parameters dueto the reasons such as (1) the sections of the target seen from eachinput section are different, and (2) the background sections (area otherthan the target area) seen from each input section are different.

Because of this, the color of one section, which should be constant fromone data input section to another, will vary. This will pose a seriousproblem when a single set of three-dimensional object data is ultimatelyproduced. However, by determining one set of reference parameters andusing the reference parameters as common parameters, images can be inputsatisfying the condition that all areas of one section will have auniform color.

Here, a case is shown where the reference parameters are automaticallydetermined at one data input section. It is also possible for anoperator to decide the reference parameters. There are also cases wherean operator wishes to set a more suitable set of parameter values otherthan the values automatically determined at the data input section,based on his experience or for a special effect. In such cases, manualsetting is preferable.

In the above example, a case is described where three-dimensional dataof the target is created. The present method can also be applied forcreating pseudo-three-dimensional shape data of the target such asQuickTime VR. Using the present method, a problem where the hue changeswhen the target is rotated and displayed can be avoided.

As described, according to the present invention, three-dimensional datafor a target can be automatically produced and provided with desiredcontent designated by a user.

1. A three-dimensional model providing apparatus for supplyingthree-dimensional data of a target to a customer, said apparatuscomprising: an input section for inputting three-dimensional data,wherein said input section comprises: a light projection section forprojecting light, said light projection section including a surfacelight source for projecting a light pattern, a light reflecting sectionfor reflecting the light pattern incident from said light projectionsection and projecting the light pattern onto the target, a target imagereflecting section for reflecting a target image formed from the lightpattern projected onto the target, a light receiving section forreceiving the target image reflected at said target image reflectingsection, and a modeling section for producing three-dimensional data ofsaid target based on the received target image; a server section formanaging the three-dimensional data; a providing section for providingthe three-dimensional data; and a communicating section forcommunicating said three-dimensional data between said input section,said server section, and said providing section.
 2. A three-dimensionalmodel providing apparatus of claim 1, further comprising: a receivingsection for receiving instruction contents from the customer; and aconverting section for modifying, processing, or converting saidthree-dimensional data based on said instruction contents.
 3. Athree-dimensional model providing apparatus of claim 1, furthercomprising: an output section for confirming, visually or by audio, theinput three-dimensional data or modified or processed three-dimensionaldata.
 4. A three-dimensional model providing apparatus of claim 3,wherein said output section is capable of three-dimensional display. 5.A three-dimensional model providing apparatus of claim 1, wherein saidinput section produces three-dimensional shape data based on image dataof the target captured from a plurality of directions.
 6. Athree-dimensional model providing apparatus of claim 1, wherein saidinput section produces color data associated with said three-dimensionalshape data based on image data of the target captured from a pluralityof directions.
 7. A three-dimensional model providing apparatus of claim1, wherein said input section produces three-dimensional shape databased on image data of said target captured from a plurality ofdirections by a plurality of cameras.
 8. A three-dimensional modelproviding apparatus of claim 1, wherein said input section producescolor data associated with said three-dimensional shape data based onimage data of said target captured from a plurality of directions by aplurality of cameras.
 9. A three-dimensional model providing apparatusof claim 2, wherein said instruction content from the customer receivedat said receiving section identifies a data format to be used when thethree-dimensional data is provided by said three-dimensional modelproviding apparatus.
 10. A three-dimensional model providing apparatusof claim 2, wherein said instruction content from the customer receivedat said receiving section is modification and processing contents to beapplied to the three-dimensional data by said three-dimensional modelproviding apparatus.
 11. A three-dimensional model providing apparatusof claim 2, wherein said instruction content from the customer receivedat said receiving section is a data providing method designating theproviding method of the three-dimensional data by said three-dimensionalmodel providing apparatus.
 12. A three-dimensional model providingapparatus of claim 2, wherein said receiving section comprises areceived content output section for outputting the instruction contentfrom the customer and the identification information for identifying thecustomer.
 13. A three-dimensional model providing apparatus of claim 12,wherein said received content output section comprises one of adisplay-at-display section, an output-by-printing section, and anoutput-to-recording-medium section.
 14. A three-dimensional modelproviding apparatus of claim 9, wherein said converting section convertssaid three-dimensional data according to said data format.
 15. Athree-dimensional model providing apparatus of claim 10, wherein saidconverting section modifies, processes, or converts saidthree-dimensional data according to said modification and processingcontents.
 16. A three-dimensional model providing apparatus of claim 1,wherein said server section manages operation progress condition of saidproviding section and said converting section, in addition to managingsaid three-dimensional data.
 17. A three-dimensional model providingapparatus of claim 1, further comprising a communicating section,wherein said apparatus communicates, via said communicating section,said three-dimensional data, said instruction contents from thecustomer, and identification information for identifying the customerto/from another three-dimensional model providing apparatus having afunction equivalent to said converting section.
 18. A three-dimensionalmodel providing apparatus of claim 1, further comprising a communicatingsection, wherein said apparatus communicates, via said communicatingsection, three-dimensional data, the content of an instruction from thecustomer, and/or identification information for identifying the customerto/from another three-dimensional model providing apparatus having afunction equivalent to said providing section.
 19. A three-dimensionalmodel providing apparatus of claim 1, wherein said providing sectioncomprises: a recording medium selecting section capable of selecting arecording medium from among one or more types of recording mediumprovided to said three-dimensional model providing apparatus in advance;and a recording medium writing section for writing three-dimensionaldata onto the recording medium.
 20. A three-dimensional model providingapparatus of claim 1, wherein said providing section comprises arecording medium writing section for writing the three-dimensional dataonto a recording medium prepared by the customer.
 21. Athree-dimensional model providing apparatus of claim 1, wherein saidproviding section comprises recording medium mailing means for mailingthe recording medium.
 22. A three-dimensional model providing apparatusof claim 1, wherein said providing section provides saidthree-dimensional data to a providing section of anotherthree-dimensional model providing apparatus via said communicatingsection.
 23. A three-dimensional model providing apparatus of claim 22,wherein said other three-dimensional model providing apparatus is adesignated apparatus designated by the customer.
 24. A three-dimensionalmodel providing apparatus of claim 22, wherein said otherthree-dimensional model providing apparatus is a database server formanaging access by the customer, in addition to accumulating andmanaging said three-dimensional data.
 25. A three-dimensional modelproviding apparatus of claim 11, wherein said three-dimensional data andunique information of the customer are communicated to/from thedesignated apparatus based on said data providing method.
 26. Athree-dimensional model providing apparatus of claim 25, wherein saiddesignated apparatus is capable of adding new information to said uniqueinformation.
 27. A three-dimensional model providing apparatus of claim25, wherein said designated device is an entertainment device.
 28. Athree-dimensional model providing apparatus of claim 26, wherein saiddesignated apparatus is an entertainment device and said new informationis a score determined by the entertainment device as a result of theentertainment.
 29. A three-dimensional model providing apparatus ofclaim 26, wherein said designated apparatus is an entertainment deviceand said new information is information for updating thethree-dimensional data.
 30. A three-dimensional model providingapparatus of claim 26, further comprising an information re-inputtingsection for re-reading the information with the new information added atsaid designated apparatus.
 31. A three-dimensional model providingapparatus of claim 25, wherein said designated apparatus is anentertainment device and the three-dimensional data recorded in saidrecording medium is used by an entertainment program of saidentertainment device.
 32. A three-dimensional model providing apparatusof claim 25, wherein said designated apparatus is a database section andmanages the input three-dimensional model and entertainment program. 33.A three-dimensional model providing apparatus of claim 32, wherein saidserver section reads a model template of entertainment characters fromsaid database section, and the input three-dimensional model is changedat said converting section based on said character model template.
 34. Athree-dimensional model providing apparatus of claim 32, wherein saidserver section reads an entertainment program from said databasesection, and the input three-dimensional model is incorporated into saidentertainment program at said converting section.
 35. Athree-dimensional model providing apparatus of claim 27, wherein aplurality of said entertainment devices exist.
 36. A three-dimensionalmodel providing apparatus of claim 27, wherein said plurality ofentertainment devices are capable of communicating with each other. 37.A three-dimensional model providing apparatus of claim 23, wherein saiddesignated apparatus is a device connected to a network including acomputer.
 38. A three-dimensional model providing apparatus forsupplying three-dimensional data of a target to a customer, saidapparatus comprising: an input section for inputting three-dimensionaldata, wherein said input section comprises: a light projection sectionfor projecting light, said light projection section including a surfacelight source for projecting a light pattern, said light projectionsection being located behind the target, a light reflecting section forreflecting the light pattern incident from said light projection sectionand projecting the light pattern onto the target, said light reflectingsection being located in front of the target, a light receiving sectionfor receiving a target image, and a modeling section for producingthree-dimensional data of said target based on the received targetimage; a server section for managing the three-dimensional data; aproviding section for providing the three-dimensional data; and acommunicating section for communicating said three-dimensional databetween said input section, server section, and providing section.
 39. Athree-dimensional model providing apparatus of claim 38, furthercomprising a target image reflecting section for reflecting the targetimage formed from the light pattern projected onto the target to thelight receiving section.
 40. A three-dimensional model providingapparatus of claim 38, further comprising: a receiving section forreceiving instruction contents from the customer; and a convertingsection for modifying, processing, or converting said three-dimensionaldata based on said instruction contents.
 41. A three-dimensional modelproviding apparatus of claim 38, further comprising an output sectionfor confirming, visually or by audio, the input three-dimensional dataor modified or processed three-dimensional data.
 42. A three-dimensionalmodel providing apparatus for supplying three-dimensional data of atarget to a customer, said apparatus comprising: an input section forinputting three-dimensional data, wherein said input section comprises:a light projection section for projecting light, said light projectionsection including a surface light source for instantaneously projectinga two dimensional light pattern, a light reflecting section forreflecting the two dimensional light pattern incident from said lightprojection section and projecting the two dimensional light pattern ontothe target, a target image reflecting section for reflecting a targetimage of a complete target formed from the two dimensional light patternprojected onto the target, a light receiving section for instantaneouslyreceiving the target image of the complete target reflected at saidtarget image reflecting section, and a modeling section for producingthree-dimensional data of said target based on the received targetimage; a server section for managing the three-dimensional data; aproviding section for providing the three-dimensional data; and acommunicating section for communicating said three-dimensional databetween said input section, server section, and providing section.
 43. Athree-dimensional model providing apparatus of claim 42, furthercomprising: a receiving section for receiving instruction contents fromthe customer; and a converting section for modifying, processing, orconverting said three-dimensional data based on said instructioncontents.